🕷️️ Job Radar • SCRAPING

upwork.com 🟡 2026-05-06

🔹 Vendor Database & Ranking System Development
👤 Client: 🇺🇸 United States Member since 2026-05-05
💰 Price: ****
🚩 Problem: Developing a dynamic vendor database and ranking system for estate planning services.
📦 Existing: [Not specified]

Specifications:

[Target] Build a backend system to aggregate, normalize, deduplicate, and rank vendors in the end-of-life and estate ecosystem.
[Method] Implement data pipelines for continuous ingestion of new listings, reviews, interactions, and performance signals.
[UI/UX] Not applicable (backend-focused).
[Stack] Python/Django or Flask, PostgreSQL, Apache Airflow, Elasticsearch, Scrapy for web scraping, AWS S3 for storage, Docker for containerization.
[Security] Ensure data privacy and security with encryption at rest and in transit. Implement access controls and audit logs.
[Format] JSON for structured data exchange; Elasticsearch for search optimization.

Workflow:

1. Define vendor schema and data model.
2. Set up data pipelines using Apache Airflow to ingest new listings, reviews, interactions, and performance signals.
3. Implement normalization and deduplication logic to ensure consistent data quality.
4. Develop ranking algorithms based on various performance metrics.
5. Integrate Elasticsearch for efficient search capabilities.
6. Deploy system using Docker containers on AWS.